Goto

Collaborating Authors

 Montgomery County


MAGPI: Multifidelity-Augmented Gaussian Process Inputs for Surrogate Modeling from Scarce Data

Rex, Atticus, Qian, Elizabeth, Peterson, David

arXiv.org Machine Learning

Supervised machine learning describes the practice of fitting a parameterized model to labeled input-output data. Supervised machine learning methods have demonstrated promise in learning efficient surrogate models that can (partially) replace expensive high-fidelity models, making many-query analyses, such as optimization, uncertainty quantification, and inference, tractable. However, when training data must be obtained through the evaluation of an expensive model or experiment, the amount of training data that can be obtained is often limited, which can make learned surrogate models unreliable. However, in many engineering and scientific settings, cheaper \emph{low-fidelity} models may be available, for example arising from simplified physics modeling or coarse grids. These models may be used to generate additional low-fidelity training data. The goal of \emph{multifidelity} machine learning is to use both high- and low-fidelity training data to learn a surrogate model which is cheaper to evaluate than the high-fidelity model, but more accurate than any available low-fidelity model. This work proposes a new multifidelity training approach for Gaussian process regression which uses low-fidelity data to define additional features that augment the input space of the learned model. The approach unites desirable properties from two separate classes of existing multifidelity GPR approaches, cokriging and autoregressive estimators. Numerical experiments on several test problems demonstrate both increased predictive accuracy and reduced computational cost relative to the state of the art.




Binary perceptron computational gap -- a parametric fl RDT view

Stojnic, Mihailo

arXiv.org Machine Learning

Recent studies suggest that asymmetric binary perceptron (ABP) likely exhibits the so-called statistical-computational gap characterized with the appearance of two phase transitioning constraint density thresholds: \textbf{\emph{(i)}} the \emph{satisfiability threshold} $α_c$, below/above which ABP succeeds/fails to operate as a storage memory; and \textbf{\emph{(ii)}} \emph{algorithmic threshold} $α_a$, below/above which one can/cannot efficiently determine ABP's weight so that it operates as a storage memory. We consider a particular parametric utilization of \emph{fully lifted random duality theory} (fl RDT) [85] and study its potential ABP's algorithmic implications. A remarkable structural parametric change is uncovered as one progresses through fl RDT lifting levels. On the first two levels, the so-called $\c$ sequence -- a key parametric fl RDT component -- is of the (natural) decreasing type. A change of such phenomenology on higher levels is then connected to the $α_c$ -- $α_a$ threshold change. Namely, on the second level concrete numerical values give for the critical constraint density $α=α_c\approx 0.8331$. While progressing through higher levels decreases this estimate, already on the fifth level we observe a satisfactory level of convergence and obtain $α\approx 0.7764$. This allows to draw two striking parallels: \textbf{\emph{(i)}} the obtained constraint density estimate is in a remarkable agrement with range $α\in (0.77,0.78)$ of clustering defragmentation (believed to be responsible for failure of locally improving algorithms) [17,88]; and \textbf{\emph{(ii)}} the observed change of $\c$ sequence phenomenology closely matches the one of the negative Hopfield model for which the existence of efficient algorithms that closely approach similar type of threshold has been demonstrated recently [87].


Codec Avatar Studio: Paired Human Captures for Complete, Driveable, and Generalizable Avatars

Neural Information Processing Systems

To create photorealistic avatars that users can embody, human modeling must be complete (encompass the full body), driveable (able to reproduce motion of the user from lightweight sensors), and generalizable ( i.e., easily adaptable to novel identities). Towards these goals, paired captures, that is, captures of the same subject obtained from systems of diverse quality and availability, are crucial. However, paired captures are rarely available to researchers outside of dedicated industrial labs: Codec Avatar Studio is our proposal to close this gap. Towards generalization and driveability, we introduce a dataset of 256 subjects captured in two modalities: high resolution multi-view scans of their heads, and video from the internal cameras of a headset.



Exploring Gender Differences in Chronic Pain Discussions on Reddit

Andrade, Ancita Maria, Banerjee, Tanvi, Mundugar, Ramakrishna

arXiv.org Artificial Intelligence

Pain is an inherent part of human existence, manifesting as both physical and emotional experiences, and can be categorized as either acute or chronic. Over the years, extensive research has been conducted to understand the causes of pain and explore potential treatments, with contributions from various scientific disciplines. However, earlier studies often overlooked the role of gender in pain experiences. In this study, we utilized Natural Language Processing (NLP) to analyze and gain deeper insights into individuals' pain experiences, with a particular focus on gender differences. We successfully classified posts into male and female corpora using the Hidden Attribute Model-Convolutional Neural Network (HAM-CNN), achieving an F1 score of 0.86 by aggregating posts based on usernames. Our analysis revealed linguistic differences between genders, with female posts tending to be more emotionally focused. Additionally, the study highlighted that conditions such as migraine and sinusitis are more prevalent among females and explored how pain medication affects individuals differently based on gender.


Fully lifted \emph{blirp} interpolation -- a large deviation view

Stojnic, Mihailo

arXiv.org Machine Learning

In [104] a powerful fully lifted (fl) probabilistic blirp interpolating mechanism was introduced. It arrived as a strong upgrade on partially lifted concepts from [100, 101] and the basic ones from [49, 84] (see also, e.g., [31, 32, 60, 106] for early considerations as well as [5, 64, 67, 101, 107] for a brief history, relevance, and development overview). While the range of applicability in a variety of scientific fields is rather wide, applications in random optimizations are of our prevalent interest. They became particularly fruitful over the last two decades (some of the most prominent examples include, compressed sensing, machine learning, and neural network statistical studies; see, e.g., [50, 72-75, 86-91, 108]). Characterizing typical behavior of their various features ranging from standard optimization metrics (objective values, optimal solutions, relations between optimizing variables) to associated algorithmic ones (accuracy, speed, convergence) became possible in large part due to a strong progress made in understanding and developing powerful comparison mechanisms. For example, many of the above performance metrics often exhibit the so-calledphase-transition (PT) phenomenon where they undergo an abrupt change as one moves from one region of system parameters to another.


A large deviation view of \emph{stationarized} fully lifted blirp interpolation

Stojnic, Mihailo

arXiv.org Machine Learning

We consider \emph{bilinearly indexed random processes} (blirp) and study their interpolating comparative mechanisms. Generic introduction of the \emph{fully lifted} (fl) blirp interpolation in [105] was followed by a corresponding stationarization counterpart in [103]. A \emph{large deviation} upgrade of [105] introduced in companion paper [106] is complemented here with the corresponding one of [103]. Similarly to [106], the mechanism that we introduce extends the range of [103]'s applicability so that it encompasses random structures \emph{atypical} features. Among others these include the \emph{local entropies} (LE) which explain atypical solutions clusterings in hard random optimization problems believed to be directly responsible for the presumable existence of the so-called \emph{computational gaps}. Moreover (and similar to [105]), despite on occasion somewhat involved technical considerations, the final forms of the uncovered fundamental interpolating parameters relations are rather elegant and as such provide a valuable tool readily available for further use.


Rare dense solutions clusters in asymmetric binary perceptrons -- local entropy via fully lifted RDT

Stojnic, Mihailo

arXiv.org Machine Learning

We study classical asymmetric binary perceptron (ABP) and associated \emph{local entropy} (LE) as potential source of its algorithmic hardness. Isolation of \emph{typical} ABP solutions in SAT phase seemingly suggests a universal algorithmic hardness. Paradoxically, efficient algorithms do exist even for constraint densities $α$ fairly close but at a finite distance (\emph{computational gap}) from the capacity. In recent years, existence of rare large dense clusters and magical ability of fast algorithms to find them have been posited as the conceptual resolution of this paradox. Monotonicity or breakdown of the LEs associated with such \emph{atypical} clusters are predicated to play a key role in their thinning-out or even complete defragmentation. Invention of fully lifted random duality theory (fl RDT) [90,93,94] allows studying random structures \emph{typical} features. A large deviation upgrade, sfl LD RDT [96,97], moves things further and enables \emph{atypical} features characterizations as well. Utilizing the machinery of [96,97] we here develop a generic framework to study LE as an ABP's atypical feature. Already on the second level of lifting we discover that the LE results are closely matching those obtained through replica methods. For classical zero threshold ABP, we obtain that LE breaks down for $α$ in $(0.77,0.78)$ interval which basically matches $α\sim 0.75-0.77$ range that currently best ABP solvers can handle and effectively indicates that LE's behavior might indeed be among key reflections of the ABP's computational gaps presumable existence.